Systems and methods for image segmentation

ABSTRACT

The present disclosure relates to an image processing method. The method may include: obtaining image data; reconstructing an image based on the image data, the image including one or more first edges; obtaining a model, the model including one or more second edges corresponding to the one or more first edges; matching the model and the image; and adjusting the one or more second edges of the model based on the one or more first edges.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of International Application No.PCT/CN2017/083184, field on May 5, 2017,the contents of which isincorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to methods and systems for imageprocessing, in particular to methods and systems for image matching andsegmentation based on a probability.

BACKGROUND

With the improvement of human living standards and the prolongation oflife expectancy, cardiovascular disease has become the leading cause ofdeath in humans. Therefore, early diagnosis of cardiovascular diseasemay effectively reduce the mortality rate. Understanding the imagingperformances and function data of cardiac structure is an importantprerequisite for the correct diagnosis of the cardiovascular disease.The development of computed tomography (CT) technology has significantlyimproved the time resolution, reduced the heartbeat artifacts, andshowed a good application potential in displaying the fine structure ofthe heart.

Image segmentation is a key technology in image analysis, which plays anincreasingly important role in imaging medicine. Image segmentation isan indispensable means to extract quantitative information of specialtissues in images and is also a pre-processing step and premise forvisualization. Segmented images are widely used in a variety ofapplications, such as quantitative analysis of tissue volume, diagnoses,localization of diseased tissues, learning of anatomical structures,treatment planning, local body effect correction of functional imagingdata, and computer-guided surgeries.

After a CT image is reconstructed, it is necessary to locate andidentify the cardiac chamber on the CT image. The positioning andidentification of the cardiac chamber requires the detection of heartedge. A variable model is commonly used in the field of segmentation ofcardiac chamber. A cardiac chamber model (i.e., the variable model)corresponds to average image data of multiple sets of clinical cardiacchamber models. The matched image is obtained by matching the cardiacchamber model with the image.

SUMMARY

According to one aspect of the present disclosure, a method isdisclosed. The method may include: obtaining image data; reconstructingan image based on the image data, wherein the image includes one or morefirst edges that have a plurality of points; obtaining a model, whereinthe model includes one or more second edges corresponding to the one ormore first edges; matching the model and the image; and adjusting theone or more second edges of the model based on the one or more firstedges.

In some embodiments, the image data may include a brain image, a skullimage, a chest image, a cardiac image, a breast image, an abdominalimage, a kidney image, a liver image, a pelvic image, a perineal image,a limb image, a spine image, or a vertebral image.

In some embodiments, the obtaining a model may include obtaining aplurality of reference models; registering the plurality of referencemodels; determining a plurality of control points on the plurality ofreference models after the registration; obtaining a control point ofthe model based on a plurality of control points on the plurality ofreference models; and generating the model based on the control point ofthe model.

In some embodiments, the method may further include generating acorrelation factor of the control point of the model based on arelationship between the control point of the model and the one or moresecond edges of the model.

In some embodiments, the adjusting one or more second edges of the modelmay include determining a reference point on the second edge;determining an object point corresponding to the reference point; andadjusting the one or more second edges of the model based on the objectpoint.

In some embodiments, the determining an object point corresponding tothe reference point may include determining a normal of the referencepoint; obtaining a step size and a search range; determining one or morecandidate points along the normal based on the step size and the searchrange; obtaining a first classifier; determining a probability that theone or more candidate points correspond to the first edge based on thefirst classifier; and determining the object point based on theprobability that the one or more candidate points correspond to thefirst edge.

In some embodiments, the determining the normal of the reference pointmay include determining one or more polygon grids adjacent to thereference point; determining one or more normals corresponding to theone or more polygon grids; and determining the normal of the referencepoint based on the one or more normals.

In some embodiments, the matching the model and the image may includeobtaining a second classifier; performing a weighted Generalized HoughTransform based on the second classifier; and matching the model and theimage based on a result of the weighted Generalized Hough Transform.

In some embodiments, the obtaining the first classifier may includeobtaining a point classifier, wherein the point classifier classifies aplurality of points of the first edge based on image features related tosharpness and location; obtaining the plurality of classified points bythe point classifier, wherein at least a portion of the plurality ofclassified points are within a certain range of the first edge;determining a plurality of classified points within the certain range ofthe first edge as positive samples; determining a plurality ofclassified points outside the certain range of the first edge asnegative samples; classifying the positive samples and the negativesamples; and obtaining the first classifier based on the classifiedpositive samples and the classified negative samples.

In some embodiments, the obtaining a second classifier may includeobtaining a plurality of points of the model, wherein at least a portionof the plurality of points are within a certain range of the secondedge; determining a plurality of points within the certain range of thesecond edge as positive samples; determining a plurality of pointsoutside the certain range of the second edge as negative samples;classifying the positive samples and the negative samples based on thesharpness and location; and obtaining the second classifier based on theclassified positive samples and the negative samples.

According to another aspect of the present disclosure, a system isdisclosed. The system may include a storage configured to store data andinstructions, and a processor in communication with the storage. Whenexecuting the instruction in the storage, the processor may beconfigured to obtain image data; reconstruct an image based on the imagedata, wherein the image includes one or more first edges that have aplurality of points; obtain a model, wherein the model includes one ormore second edges corresponding to the one or more first edges; matchthe model with the image; and adjust the one or more second edges of themodel based on the one or more first edges.

According to another aspect of the present disclosure, a non-transitorycomputer-readable medium having computer programs is disclosed. Thecomputer program product may include a plurality of instructions. Theplurality of instructions may be configured to perform a method: themethod includes: obtaining image data; reconstructing an image based onthe image data, wherein the image includes one or more first edges thathave a plurality of points; obtaining a model, wherein the modelincludes one or more second edges corresponding to the one or more firstedges; matching the model and the image; and adjusting the one or moresecond edges of the model based on the one or more first edges.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to illustrate the technical solutions related to someembodiments of the present disclosure, brief descriptions of thedrawings used in the present disclosure are provided below. Obviously,drawings described below are only some examples or embodiments of thepresent disclosure. Those having ordinary skills in the art, withoutfurther creative efforts, may apply the present disclosure to othersimilar scenarios according to the drawings. Unless stated otherwise orobvious from the context, the same reference numeral in the drawingsrefers to the same structure and operation.

FIG. 1 is a schematic diagram illustrating an exemplary applicationscenario of a control and processing system according to someembodiments of the present disclosure;

FIG. 2 is a schematic diagram illustrating an exemplary systemconfiguration of a processing device according to some embodiments ofthe present disclosure;

FIG. 3 is a schematic diagram illustrating an exemplary mobile devicefor implementing some specific systems according to some embodiments ofthe present disclosure;

FIG. 4 is a schematic diagram illustrating an exemplary processingdevice according to some embodiments of the present disclosure;

FIG. 5 is a flowchart illustrating an exemplary process for implementingthe processing device according to some embodiments of the presentdisclosure;

FIG. 6 is a schematic diagram illustrating an exemplary modelconstruction module according to some embodiments of the presentdisclosure;

FIG. 7 is a flowchart illustrating an exemplary process for constructingan average model according to some embodiments of the presentdisclosure;

FIG. 8 is a schematic diagram illustrating an exemplary training moduleaccording to some embodiments of the present disclosure;

FIG. 9 is a flowchart illustrating an exemplary process for training aclassifier according to some embodiments of the present disclosure;

FIG. 10 is a schematic diagram illustrating an exemplary model matchingmodule according to some embodiments of the present disclosure;

FIG. 11 is a flowchart illustrating an exemplary process for matching anaverage model and a reconstructed image according to some embodiments ofthe present disclosure;

FIG. 12 is a schematic diagram illustrating an exemplary adjustmentmodule according to some embodiments of the present disclosure;

FIG. 13 is a flowchart illustrating an exemplary process for adjusting amodel according to some embodiments of the present disclosure;

FIG. 14 is a flowchart illustrating an exemplary process for determiningan object point according to some embodiments of the present disclosure;

FIG. 15 is a flowchart illustrating an exemplary process for determininga normal of an average model edge point according to some embodiments ofthe present disclosure;

FIG. 16 is a flowchart illustrating an exemplary process fortransforming an average model edge point according to some embodimentsof the present disclosure;

FIG. 17 is a schematic diagram illustrating an image sharpness accordingto some embodiments of the present disclosure;

FIG. 18 is an embodiment illustrating an image edge classificationtraining according to some embodiments of the present disclosure;

FIG. 19 is an embodiment illustrating a model grid classificationaccording to some embodiments of the present disclosure;

FIG. 20 is an embodiment illustrating a model grid partitioningaccording to some embodiments of the present disclosure;

FIG. 21 is an embodiment illustrating a correlation factor-based gridmodel according to some embodiments of the present disclosure;

FIG. 22 is a diagram illustrating an image edge based on the sharpnessclassification according to some embodiments of the present disclosure;

FIG. 23 is a diagram illustrating an exemplary model based on thesharpness classification according to some embodiments of the presentdisclosure;

FIG. 24 is an embodiment illustrating an image probability map obtainedbased on a classifier according to some embodiments of the presentdisclosure;

FIG. 25 is a diagram illustrating an average grid model and an imagematching after Hough transform according to some embodiments of thepresent disclosure;

FIG. 26 is a diagram illustrating an exemplary segmentation result of anadjusted precisely matched image chamber according to some embodimentsof the present disclosure;

FIG. 27A shows an image segmentation image divided not based on acorrelation factor according to some embodiments of the presentdisclosure;

FIG. 27B shows an image segmentation image divided based on acorrelation factor according to some embodiments of the presentdisclosure.

DETAILED DESCRIPTION

As used in the disclosure and the appended claims, the singular forms“a,” “an,” and “the” may be intended to include the plural forms aswell, unless the context clearly indicates otherwise. It will be furtherunderstood that the terms “include” and/or “comprise,” when as usedherein, specify the presence of operations and/or elements, but do notexclude the presence or addition of one or more other operations and/orelements thereof. The term “based on” is “based at least in part on.”The term “one embodiment” means “at least one embodiment;” the term“another embodiment” means “at least one additional embodiment.” Therelevant definitions of other terms will be given in the descriptionbelow.

The flowcharts used in the present disclosure illustrate operations thatsystems implement according to some embodiments of the presentdisclosure. It is to be expressly understood, the operations of theflowcharts may be implemented not in order. Conversely, the operationsmay be implemented in an inverted order, or simultaneously. Moreover,one or more other operations may be added to the flowcharts. One or moreoperations may be removed from the flowcharts.

The present disclosure relates to a process of image segmentation, whichmatches an edge of an image based on a probability and transforms anedge point of a model based on a grid model of a correlation factor toachieve an accurate matching or segmentation. The matching of a medicalimage may include seeking one or more spatial transformations for themedical image to make the medical image spatially consistent withcorresponding points on the model. The consistency may include the sameanatomical point on the human body having the same spatial position onthe matched image and model. The matching may result in a match betweenall anatomical points on the image and/or all diagnostically significantanatomical points and points of interest (POI) for the surgery.

FIG. 1 is a schematic diagram illustrating an exemplary applicationscenario of a control and processing system according to someembodiments of the present disclosure. As shown in FIG. 1, the controland processing system 100 (also referred to as system 100) may includean imaging device 110, a database 120, and a processing device 130.

The imaging device 110 may generate an image by scanning a targetobject. The image may be or include a variety of medical images, such asa head image, a chest image, an abdominal image, a pelvic image, aperineal image, a limb image, a spine image, a vertebra image, or thelike, or any combination thereof. The head image may include a brainimage, a skull image, or the like. The chest image may include an entirechest image, a cardiac image, a breast image, or the like. The abdominalimage may include an entire abdominal image, a kidney image, a liverimage, or the like. The cardiac image may include, but not be limitedto, a omni-directional digitized cardiac image, a digitized cardiactomography, a cardiac phase-contrast image, an X-ray image, amultimodality image, or the like. The medical image may be atwo-dimensional image, a three-dimensional image, or the like. Theformat of the medical image may include a JPEG format, a TIFF format, aGIF format, an FPX format, or the like. The medical image may be storedin the database 120, or may be transmitted to the processing device 130for image processing. The cardiac image may be taken as an example insome embodiments of the present disclosure, but those skilled in the artwill understand that the system and method disclosed in the presentdisclosure may also be used for other images.

The database 120 may store image(s) and/or information relating to theimage(s). The image(s) and the information relating to the image(s) maybe provided by the imaging device 110 and/or the processing device 130,or may be obtained outside the system 100, for example, informationinputted by a user, information obtained from a network, or the like.The information relating to the image(s) may include an algorithm, asample, a model, a parameter for processing the image(s), real-time dataduring the processing, or the like. The database 120 may be ahierarchical database, a network database, a relational database, or thelike. The database 120 may be a local database, or a remote database.The database 120 or other storage devices within the system may digitizethe information and then store the digitized information using a storagedevice that operates in an electrical manner, in an optical manner, orin a magnetical manner. In some embodiments, the database 120 or otherstorage devices within the system may be a device that storesinformation using electrical energy, such as a random access memory(RAM), a read-only memory (ROM), or the like. The random access memorymay include, but not be limited to, a decimal counter tube, a countingtube, a delay line memory, a Williams tube, a dynamic random accessmemory (DRAM), a static random access memory (SRAM), a thyristor randomaccess memory (T-RAM), a zero capacitance random access memory (Z-RAM),or the like, or any combination thereof. The read-only memory mayinclude, but not be limited to, a magnetic bubble memory, a magneticbutton line memory, a thin-film memory, a magnetic plate line memory, amagnetic core memory, a drum memory, an optical disk drive, a hard disk,a magnetic tape, a non-volatile memory (NVRAM), a phase change memory, amagnetoresistive random storage memory, a ferroelectric random accessmemory, a non-volatile static random access memory, a programmableread-only memory, a shielded heap read memory, a floating connectiongate random access memory, a nano random access memory, a track memory,a variable resistive memory, a programmable metallization unit, or thelike, or any combination thereof. In some embodiments, the database 120or the other storage devices within the system may be a device thatstores the information in a magnetic manner, such as a hard disk, afloppy disk, a magnetic tape, a magnetic core memory, a magnetic bubblememory, a USB flash drive, a memory, or the like. In some embodiments,the database 120 or the other storage devices within the system may be adevice that stores the information in an optical manner, such as acompact disc (CD), a digital video disc (DVD), or the like. In someembodiments, the database 120 may be a device that stores theinformation in a magneto-optical manner, such as a magneto-optical disk.The access mode of the database 120 or the other storage devices withinthe system may include a random access, a serial access, a read-onlyaccess, or the like, or any combination thereof. The database 120 or theother storage devices within the system may be a transitory memory or anon-transitory memory. The storage devices mentioned above are someexamples for illustration purposes, and are not intended to limit thescope of the present disclosure. The database 120 may be any suitablestorage devices.

The database 120 may be a portion of the processing device 130, or maybe a portion of the imaging device 110, or may be independent of theprocessing device 130 and/or the imaging device 110. In someembodiments, the database 120 may be connected to other modules ordevices within the system 100 via the network 150. The connection mayinclude a wired connection, a wireless connection, or a combinationthereof.

The processing device 130 may obtain the image (or image data) from theimaging device 110, or the database 120. The processing device 130 mayprocess the obtained image. The processing of the image may includegrayscale histogram processing, normalization processing, geometrictransformation, spatial transformation, image smoothing processing,image enhancement processing, image segmentation processing, imagetransformation processing, image restoration, image compression, imagefeature extraction, or the like. The processing device 130 may store theprocessed image (or image data) using the database 120, or may transmitthe processed image data to a device outside the control and processingapparatus system 100.

In some embodiments, the processing device 130 may include one or moreprocessors, storages, or the like. For example, the processing device130 may include a central processor (CPU), an application-specificintegrated circuit (ASIC), a dedicated instruction set processor (ASIP),an image processor (GPU), a physical operation processor (PPU), adigital signal processor (DSP), a field-programmable gate array (FPGA),a programmable logic device (PLD)), a controller, a micro control unit,a processor, a microprocessor, an advanced RISC machine processor, orthe like, or any combination thereof.

In some embodiments, the control and processing system 100 may alsoinclude a terminal device 140. The terminal device 140 may communicatewith the imaging device 110, the database 120, and the processing device130. For example, the terminal device 140 may obtain the processed image(or image data) from the processing device 130. In some embodiments, theterminal device 140 may obtain the image data from the imaging device110 and transmit the image data to the processing device 130 for imageprocessing. The terminal device 140 may include one or more inputdevices, a control panel, or the like. For example, the input device(s)may include a keyboard, a touch screen, a mouse, a voice input device, ascan device, an information recognition device (such as a human eyerecognition system, a fingerprint recognition system, a brain monitoringsystem, etc.), a remote controller, or the like.

The control and processing system 100 may be connected to the network150. The network 150 may be a wireless network, a mobile network, awired network, or the like. The wireless network may include aBluetooth, a wireless local area network (WLAN), Wi-Fi, WiMax, or thelike. The mobile network may include a 2G signal, a 3G signal, a 4Gsignal, or the like. The wired network may include a local area network(LAN), a wide area network (WAN), a proprietary network, etc.

The database 120 and the processing device 130 of the control andprocessing system 100 may execute operational instructions through acloud computing platform. The cloud computing platform may include astorage-based cloud platform for data storage, a computing-based cloudplatform for data processing, or an integrated cloud computing platformthat performs computing processing and data storage. For example, imagedata generated by the control and processing system 100 may be computedor stored by the cloud computing platform.

It should be noted that the above descriptions of the control andprocessing system 100 are merely for the convenience of illustration andare not intended to limit the scope of the present disclosure.

FIG. 2 is a schematic diagram illustrating an exemplary systemconfiguration of a processing device according to some embodiments ofthe present disclosure. As shown in FIG. 2, the processing device 130may include a data bus 210, a processor 220, a read-only storage (ROM)230, a random storage (RAM) 240, a communication port 250, aninput/output port (I/O) 260, a hard disk 270, and a display 280connected to the I/O 260. The connection manner between the hardware ofthe processing device 130 may be a wired connection, a wirelessconnection, or a combination thereof. The hardware may be local, remote,or a combination thereof.

The data bus 210 may be used to transmit data information. In someembodiments, the data transmission between the hardware of theprocessing device 130 may be implemented via the data bus 210. Forexample, the processor 220 may send the data, through the data bus 210,to other hardware such as the storage or the I/O 260. It should be notedthat the data may be real data, instruction codes, state information, orcontrol information, or the like. In some embodiments, the data bus 210may be an industry standard (ISA) bus, an extended industry standard(EISA) bus, a video electronic standard (VESA) bus, an externalcomponent interconnect standard (PCI) bus, or the like.

The processor 220 may be used for logic computations, data processing,and/or instruction generation. In some embodiments, the processor 220may obtain data/instructions from an internal memory, which may includea read-only memory (ROM), a random access memory (RAM), a cache (notshown in the figure), or the like. In some embodiments, the processor220 may include a plurality of sub-processors, which may be used toimplement different functions of the system 100.

The read-only memory (ROM) 230 may be used for power-on self-test of theprocessing device 130, the initialization of each function module of theprocessing device 130, the driver of the basic input/output of theprocessing device 130, or the like. In some embodiments, the read-onlymemory may include a programmable read-only memory (PROM), aprogrammable erasable read-only memory (EPROM), or the like. The randomaccess memory (RAM) 240 may be used to store an operation system,various applications, data, or the like. In some embodiments, the randomaccess memory (RAM) 240 may include a static random access memory(SRAM), a dynamic random access memory (DRAM), or the like.

The communication port (COM PORT) 250 may be used to connect theoperation system with an external network to implement communicationbetween the operation system and the external network. In someembodiments, the communication port 250 may include a file transferprotocol (FTP) port, a hypertext transport protocol (HTTP) port, or adomain name server (DNS) port. The input/output port 260 may be used forthe exchanging and controlling of data and/or information between theexternal device or circuit and the processor 210. In some embodiments,the input /output port 260 may include a universal serial bus (USB)port, a peripheral interface controller (PCI) port, an integrateddevelopment environment (IDE) port, etc.

The hard disk 270 may be used to store the information and the datagenerated by the processing device 130 or received from the processingdevice 130. In some embodiments, the hard disk 270 may include amechanical hard disk (HDD), a solid-state hard disk (SSD), or a hybridhard disk (HHD). The display 280 may be used to display the informationand the data generated by the system 130 to a user. In some embodiments,the display 280 may include a physical display, such as a display with aspeaker, a liquid crystal display (LCD), alight emitting diode (LED)display, an organic light emitting diode (OLED) display, an electronicink (E-Ink) display, or the like.

FIG. 3 is a schematic diagram illustrating an exemplary mobile devicefor implementing some specific systems according to some embodiments ofthe present disclosure. As shown in FIG. 3, the mobile device 300 mayinclude a terminal device 140. In some embodiments, a user may receiveor transmit information related to the control and processing system 100via the mobile device 300. The mobile device 300 may include asmartphone, a personal digital assistant (PDA), a tablet computer, ahandheld game player, a smart glasses, a smartwatch, a wearable device,a virtual reality device, or a display enhancement device, or the like,or any combination thereof. In some embodiments, the mobile device 300may include one or more central processing units (CPUs) 340, one or moregraphics processing units (GPUs) 330, a display 320, a memory 360, acommunication platform 310, a storage 390, and one or more input/outputdevices (I/O) 350. In some embodiments, the mobile device 300 may alsoinclude a system bus, a controller, or the like. As shown in FIG. 3, theCPU may download the mobile device operation system (e.g., iOS, Android,Windows Phone, etc.) 370 and one or more applications 380 to the memory360 from the storage 390. The one or more applications 380 may include aweb page or other mobile application software (App) for receiving andtransmitting information related to the control and processing system100. The user may obtain or provide the information via the input/outputdevice 350, which may be further transmitted to the control andprocessing system 100 and/or device units in the system.

In some embodiments of the present disclosure, the computer hardwareplatform may be used as a hardware platform for one or more elements(e.g., the control and processing system 100 and other parts thereof),implementing various modules, units, and functions thereof. The hardwareelements, operation systems, and programming language may be inherentlytraditional, and those skilled in the art may be likely to adapt thesetechniques to cardiac image model construction and edge segmentation. Acomputer with a user interface may be a personal computer (PC), otherworkstations or terminal devices. A properly programmed computer mayalso be used as a server. Since those skilled in the art may be familiarwith the structure, the programming and the general operation of thecomputer device used in the present disclosure, no further explanationwill be given for other drawings.

FIG. 4 is a schematic diagram illustrating an exemplary processingdevice according to some embodiments of the present disclosure. Theprocessing device 130 may include an obtaining module 410, an imagereconstruction module 420, a storage module 430, a model constructionmodule 430, a training module 450, a matching module 460, and anadjustment module 470. The connection between modules of the processingdevice 130 may be a wired connection, a wireless connection, or acombination thereof. Any module may be local, remote, or a combinationthereof.

The storage module 430 may be configured to store image data orinformation, whose functions may be implemented by one or morecombination of the hard disk 270, the read-only storage (ROM) 230, therandom access storage (RAM) 240 in FIG. 2. The storage module 430 maystore information of other modules within the processing device 130, orinformation of modules or devices outside the processing device 130. Theinformation stored by the storage module 430 may include scan data ofthe imaging device 110, control commands or parameters inputted by theuser, intermediate data or complete data generated by the processingdevice 130. In some embodiments, the storage module 430 may send thestored information to one or more modules of the processing device 130(e.g., the image reconstruction model 420, the model construction model430, etc.) for image processing. In some embodiments, the storage module430 may store the information generated by the one or more modules ofthe processing device 130, such as real-time calculation data. Thestorage module 430 may include, but not be limited to, various types ofstorage devices such as a solid-state hard disk, a mechanical hard disk,a USB flash memory, an SD memory card, an optical disk, a random storage(RAM), or a read-only storage (ROM), or the like. The storage module 430may be a storage device of the system, or may be a storage deviceoutside the system or an external storage device, such as a storage onthe cloud storage server.

The obtaining module 410 may be configured to obtain the image datacollected by the imaging device 110, the image data stored by thedatabase 120, or the data outside the control and processing system 100.The functions of the obtaining module 410 may be implemented by theprocessor 220 in FIG. 2. The image data may include the image datacollected by the imaging device 110, an algorithm, a sample, a model, ora parameter for processing an image, real-time data during theprocessing, or the like. In some embodiments, the obtaining module 410may send the obtained image data or information to the imagereconstruction module 420 for processing. In some embodiments, theobtaining module 410 may send the obtained algorithm, the parameter,etc. of the processed image to the model construction module 440. Insome embodiments, the obtaining module 410 may send the obtained imagedata or information to the storage module 370 for storage. In someembodiments, the obtaining module 410 may send the obtained sample, theparameter, the model, the real-time data, etc. to the training module450, the matching module 460, or the adjustment module 470. In someembodiments, the obtaining module 410 may receive a data obtaininginstruction from the processor 220 and perform the corresponding dataobtaining operation. In some embodiments, the obtaining module 410 maypreprocess the image data or information after obtaining the image dataor information.

The image reconstruction module 420 may be configured to reconstruct amedical image, whose functions may be implemented by the processor 220in FIG. 2. In some embodiments, the image reconstruction module 420 mayobtain the image data or information from the obtaining module 410 orthe storage module 430, and reconstruct the medical image based on theimage data or information. The medical image may be a three-dimensionalmedical image of a human body. The image data may include scan dataobtained at different times, different positions, and different angles.According to the scan data, the image reconstruction module 420 maycompute feature or state of a portion of the human body, such as anabsorption capacity of the corresponding portion of the human body, adensity of the tissue of the corresponding portion of the human body, orthe like. Then the image reconstruction module 420 may reconstruct thethree-dimensional medical image of the human body. In some embodiments,the three-dimensional medical image of the human body may be displayedvia the display 280 or stored by the storage module 430. In someembodiments, the three-dimensional medical image of the human body maybe transmitted to the model construction module 440 for furtherprocessing.

The model construction module 440 may be configured to construct athree-dimensional average model of a target object. In some embodiments,the target object may be a heart. The three-dimensional average modelmay be a three-dimensional average grid model of a heart chamberconstructed based on a plurality of sets of reference models. In someembodiments, the model construction module 440 may obtain the referencemodel(s) of at least one heart chamber and information related to thereference model(s) from the obtaining module 410, the storage module430, or the user input. The information related to the referencemodel(s) may include a size of an image, pixels of the image, a spatialposition of the pixel(s), or the like. In some embodiments, the modelconstruction module 440 may perform registration (or pre-processing) onthe reference model(s) based on the obtained reference model(s) of theat least one heart chamber and the information related to the referencemodel(s), so that the directions and ratios of all the reference modelsare consistent. The edge of the chamber of the pre-processed image maybe labelled manually or automatically labelled by the processor. Thenthe reference model (e.g., heart reference model) may be divided into aplurality of sub-heart chambers, and a heart chamber average grid modelmay be constructed based on the edge point data of each chamber. Themodel construction module 440 may send the heart chamber average gridmodel to the storage module 430 for storage, or to the training module450 or the matching module 460 for further processing. In someembodiments, the model construction module 440 may also determine therelationship between one or more chambers in the average model (e.g.,the heart chamber average grid model) based on the plurality of sets ofthe reference models. For example, the modelconstruction module 440 mayconstruct a correlation factor matrix that represents the effect of eachchamber on one or more edge data points. By constructing the correlationfactor matrix, the chamber boundary separation may be improved. Themodel construction module 440 may send the constructed correlationfactor matrix to the storage module 430 for storage, or to the matchingmodule 460 or the adjustment module 470 for further processing.

The training module 450 may be configured to train a classifier. Thetraining module 450 may divide possible edge points into differentchamber categories. For example, the training module 450 may divide arange of data points near the edge of the reference model intosix-chamber categories, such as the left ventricle, the left atrium, theright ventricle, the right atrium, the left myocardium, or the aorta,respectively. As another example, the training module 450 may divide,based on the change degree of the edge of the chamber, a range of datapoints near the edge of the reference model into ten chamber categories,such as the left ventricular edge, the left atrium sharp edge, the leftatrium non-sharp edge, the right ventricular sharp edge, the rightventricle non-sharp edge, the right atrium sharp edge, the right atriumnon-sharp edge, the aortic edge, the left myocardium sharp edge, and theleft myocardial non-sharp edge, respectively. In some embodiments, thetraining module 450 may obtain the reference model(s) of the at leastone heart chamber and information related to the reference model(s) fromthe storage module 430, the model construction module 440, or the userinput. The information related to the reference model may include theedge point data of each chamber in the reference model, or the like. Insome embodiments, the training module 450 may divide the points near theedge of the chamber into positive samples and negative samples accordingto the distance between the points and the edge of the chamber. In someembodiments, the positive samples may include data points within acertain threshold range of the edge of the chamber, and the negativesamples may include data points that are distant from the edge of thechamber and located at other random positions in the space. In someembodiments, the training module 450 may train the points near the edgeof the chamber on the reference model or the average model based on thepositive sample points or the negative sample points to obtain one ormore classifiers. For example, the training module 450 may obtain apoint classifier. The point classifier may classify the plurality ofpoints of the first edge according to an image feature. The imagefeature may be related to sharpness and location. As another example,the training module 450 may obtain a first classifier. The firstclassifier may be related to the point classifier. In some embodiments,the first classifier may determine a plurality of classified pointswithin a certain range of the first edge as positive samples anddetermine a plurality of classified points outside the certain range ofthe first edge as negative samples based on the plurality of pointsclassified by the point classifier. Then, the positive samples and thenegative samples may be classified. The trained first classifier may beobtained based on the classified positive samples and the classifiednegative samples. In some embodiments, the training module 450 may trainthe classifier using Probabilistic Boosting-Tree (PBT). The PBT mayinclude a two-level PBT algorithm or a multi-level PBT algorithm. Thetraining module 450 may send the trained classifier to the storagemodule 430 for storage, or to the adjustment module 470 for furtherprocessing.

The matching module 460 may be configured to match the image (e.g., themedical image) with the average model contructed by the modelconstruction module 440 to construct a three-dimensional grid modelcorresponding to the image. The image may be obtained from the imagereconstruction module 420 or the storage module 430. In someembodiments, the matching module 460 may match the average model to theimage using, e.g., a Hough transform, to obtain a three-dimensional gridmodel of the heart chamber roughly matching the image. The matchingmodule 460 may obtain the information (such as parameters) required bythe Hough transform from the obtaining module 410, the storage module430, or the user input. The matching module 460 may send the matchedthree-dimensional grid model of the heart chamber to the storage module430 for storage, or to the adjustment module 470 for furtheroptimization.

The adjustment module 470 may be configured to optimize the model suchthat the model is closer to the real heart (e.g., cardiac image data).The adjustment module 470 may obtain the matched grid model of the heartchamber from the matching module 460 or the storage module 430. In someembodiments, the adjustment module 470 may determine the optimal heartchamber edge based on a probability that the data points within thecertain range of the edge of the chamber on the matched heart model(e.g., the matched grid model) belong to the edge of the chamber. Theadjustment module 470 may further precisely adjust the three-dimensionalgrid model of the heart chamber. The precise adjustment may include asimilarity transformation, a segmentation affine transformation, amicro-variation based on an energy function, or the like. In someembodiments, the adjustment module 470 may perform an image formconversion on the precisely adjusted three-dimensional grid model of theheart chamber to obtain an edge segmentation image of the heart chamber(as shown in FIG. 26). The adjustment module 470 may send the preciselyadjusted three-dimensional grid model of the heart chamber or the edgesegmentation image of the heart chamber to the storage module 430 forstorage, or to the display 280 for display.

It should be noted that the above descriptions of the processing device130 are merely for convenience of illustration and are not intended tolimit the scope of the present disclosure. It should be understood that,for those skilled in the art, after understanding the working principleof the device, it is possible to arbitrarily combine the variousmodules, or to connect a subsystem to other modules, or make variousmodifications and changes to the forms and details of the device withoutdeparting from the principle. For example, the model construction module440 and/or the training module 450 may be omitted or combined with thestorage module 430. All such variations are within the protection scopeof the present disclosure.

FIG. 5 is a flowchart illustrating an exemplary process for implementinga processing device according to some embodiments of the presentdisclosure. In 510, image data may be obtained. In some embodiments, theoperation 510 may be implemented by the obtaining module 410. The imagedata may be obtained from the imaging device 110, the database 120, or adevice external to the control and processing system 100. The image datamay include original image data obtained by a computed tomography (CT)device, a positron emission tomography (PET) device, a single-photonemission tomography (SPECT) device, a magnetic resonance imaging (MRI)device, an ultrasound device, and other medical image devices. In someembodiments, the image data may be original image data of the heart or aportion of the heart. In some embodiments, the operation 510 may includepre-processing the obtained original image data (e.g., cardiac originalimage data), and transmitting the pre-processed original image data tothe image reconstruction module 420 or the storage module 430 of theprocessing device 130. The pre-processing of an image may includedistortion corrections, denoising, smoothing, enhancements, etc. of theimage.

In 520, a cardiac image may be reconstructed based on the cardiac imagedata. The operation 502 may be implemented by the image reconstructionmodule 420 of the processing device 130 based on image reconstructiontechniques. The cardiac image data may be obtained from the obtainingmodule 410 or the storage module 430. The cardiac image may include aomni-directional digitized cardiac image, a digitized cardiactomography, a cardiac phase-contrast image, an X-ray image, amultimodality image, or the like. The cardiac image may be atwo-dimensional image or a three-dimensional image. The format of thecardiac image may include a JPEG format, a TIFF format, a GIF format, anFPX format, or the like. The image reconstruction technique may includea solving simultaneous equation technique, a Fourier transformreconstruction technique, a direct back projection reconstructiontechnique, a filter back projection reconstruction technique, a Fourierback projection reconstruction technique, a convolution back projectionreconstruction technique, an iterative reconstruction technique, or thelike. In some embodiments, operation 520 may include pre-processing theobtained cardiac image data and obtaining a plurality of heart sectionsor projection. In some embodiments, the obtained cardiac image data orthe pre-processed cardiac image data may include a plurality of heartsections. The image reconstruction module 420 may reconstruct thecardiac image or model based on the information provided by theplurality of the heart sections. The information provided by the heartsections may include tissue densities of different parts of the heart,absorption abilities to radiation, or the like. The reconstructedcardiac image may be displayed by the display 280 or be stored by thestorage module 430. The reconstructed cardiac image may be furtherprocessed by the model construction module 440 of the processing device130.

In 530, a three-dimensional cardiac average grid model may beconstructed. The operation 530 may be implemented by the modelconstruction module 440 of the processing device 130 based on aplurality of reference models. The operation 530 may include obtainingthe plurality of reference models from the obtaining module 410, thestorage module 430, or the user input. In some embodiments, theoperation 530 may include image registrations on the plurality ofreference models. The image registration may include a grayscale-basedimage registration, a domain transformation-based image registration, afeature-based image registration, or the like. The feature may include afeature point, a feature region, a feature edge, or the like. In someembodiments, the plurality of reference models may be heart chambersegmentation data or the reference models in which the edge of thechamber may be labelled by the user. The average model may include aPoint Distribution Model (PDM), an Active Shape Model (ASM), an ActiveContour Model (also known as Snakes), an Active Appearance Model (AAM),or other computational model, or the like. In some embodiments, theoperation 530 may include determining a relationship between therespective chambers on the constructed average model based on thechamber edge data on the plurality of reference models, and establishinga two-dimensional correlation factor matrix. In some embodiments, theprocessor 220 may send the three-dimensional cardiac average grid modelor the average model containing the correlation factor information tothe storage module 430 of the processing device 130 for storage, or tothe matching module 460 for further processing.

In 540, the cardiac image data may be matched with the three-dimensionalcardiac average grid model. In some embodiments, the matching mayinclude matching a first edge of the cardiac image data with a secondedge of the three-dimensional heart average grid model. In someembodiments, the first edge may include an outer edge and an inner edge.The outer edge may be the outer contour of the heart. The inner edge maybe the inner chamber contour of the heart. The outer contour and theinner chamber contour may be filled by heart tissues. In someembodiments, corresponding to the first edge, the second edge of thethree-dimensional cardiac average grid model may include an outer edgeand an inner edge. The outer edge of the second edge may correspond tothe edge of the outer contour of the heart, and the inner edge of thesecond edge may correspond to the edge of the contour of the innerchamber of the heart. The outer and inner edges may refer to edges for arough matching and a precise matching, respectively. If the processdisclosed in the present disclosure is used for other objects, organs ortissues, the outer and inner edges may not necessarily have ageometrical internal and external relationship. For example, for certainobjects, organs, or tissues, the edge for the rough matching may be onthe outside, inside, or the same side of the edge used for the precisematching. As another example, the edge for the rough matching mayoverlap or intersect with the edge for the precise matching. In someembodiments, the matching between the cardiac image data and thethree-dimensional cardiac average model may be a matching between theouter edge of the first edge of the cardiac image data and the outeredge of the second edge of the three-dimensional cardiac average model.In some embodiments, the operation 540 may be implemented by thematching module 460 based on an image matching algorithm. The imagematching algorithm may include a matching algorithm based on NNDR, asearch algorithm of adjacent feature points, an object detection basedon Hough transform, or the like. In some embodiments, the heart averagemodel constructed by the model construction module 440 may be matched tothe first edge on the cardiac image data obtained by the imagereconstruction module 420 based on a Generalized Hough Transform, and amatched heart model may be obtained. In some embodiments, the first edgemay include a plurality of points. The plurality of points of the firstedge may be classified based on the image feature to obtain a pointclassifier. The image feature may be related to sharpness and location.In some embodiments, a weighted Generalized Hough Transform may beimplemented based on the probability that the points on the cardiacimage data to be matched are part of the edge. The probability may becomputed by inputting each point on the cardiac image data to be matchedinto the first classifier trained by the training module 450. The firstclassifier may be obtained based on a point classifier. The pointclassifier may be determined by classifying the plurality of points ofthe first edge according to the image feature. In some embodiments, anedge probability map of the heart to be matched may be constructed basedon the probability of the points on the heart to be matched. The edgeprobability map may include a grayscale gradient map, a color gradientmap (as shown in FIG. 24), or the like. In some embodiments, the cardiacimage may be pre-processed before computing the probability that eachpoint on the cardiac image data to be matched is at an edge. Forexample, a portion of the heart that is completely impossible to be theedge of the heart may be excluded, thereby reducing the computationamount of the classifier. For example, for a CT image, the CT value ofthe muscular tissue may be generally greater than −50, and the portionwhose CT value is less than −50 may be labelled by a mask so that theclassifier may not need to compute the points within the portionlabelled by the mask. In some embodiments, the matching module 460 ofthe processing device 130 may send the matched heart model or thethree-dimensional heart grid model to the storage module 430 forstorage, or to the adjustment module 470 for further optimization.

In 550, a precisely adjusted heart chamber segmentation image may beobtained. The operation 550 may be implemented by the adjustment module470 of the processing device 130. In some embodiments, the adjustmentmodule 470 may adjust the edge points of the chamber (e.g., the inneredge of the second edge) on the model to achieve the matching with theinner edge of the first edge in the cardiac image data. In someembodiments, the operation 550 may include determining an edge objectpoint based on a chamber edge on the matched three-dimensional heartgrid model. In some embodiments, the edge object point may be determinedaccording to the probability of a second edge point within a certainrange of the chamber edge on the matched three-dimensional heart gridmodel. In some embodiments, the probability may be computed based on asecond classifier trained by the second edge points. In someembodiments, the probability may be computed based on the firstclassifier trained by the plurality of reference models or the averagemodel. In some embodiments, the operation 550 may include performing adeformation on the three-dimensional cardiac grid model based on thedetermined edge object points, thereby obtaining an adjustedthree-dimensional cardiac grid model whose chamber edge is adjusted. Thetransformation may include a similarity transformation, an affinetransformation, other image micro-deformation techniques, or the like.For example, in some embodiments, the similarity transformation, thesegmentation affine transformation, and/or the energy function-basedmicro-variation may be performed based on the determined edge objectpoints. In some embodiments, the adjustment module 470 of the processingdevice 130 may convert the adjusted three-dimensional heart grid modelinto a heart chamber segmentation image (as shown in FIG. 26) by a mask.The different chambers of the chamber segmentation image may be labelledwith different colors. In some embodiments, the adjustment module 470 ofthe processing device 130 may send the precisely adjusted heart chambermodel or the heart chamber segmentation image to the storage module 430for storage, or to the display 280 for display.

It should be noted that the above descriptions of the chambersegmentation process by the processing device 130 are merely forconvenience of illustration and are not intended to limit the scope ofthe present disclosure. It should be understood that, for those skilledin the art, after understanding the working principle of the device, itis possible to arbitrarily adjust the order of the operations, or add orremove some operation without departing from the principle. For example,the operation 530 of constructing the average model may be removed. Asanother example, the adjustment module 470 may perform one or more ofthe adjustments on the grid model, or adopt other forms ofmicro-variation. All such variations are within the protection scope ofthe present disclosure.

FIG. 6 is a schematic diagram illustrating an exemplary modelconstruction module according to some embodiments of the presentdisclosure. The model construction module 440 may include an obtainingunit 610, a registration unit 620, a labelling unit 630, an averagemodel generation unit 640, and a correlation factor generation unit 650.The connection between modules of the model construction module 440 maybe a wired connection, a wireless connection, or a combination thereof.Any module may be local, remote, or a combination thereof.

The obtaining unit 610 may be configured to obtain a plurality ofreference models. The obtaining unit 610 may obtain the information fromthe database 120, storage devices external to the controlling andprocessing system 100, or the user input. The functions of the obtainingunit 610 may be implemented by the processor 220 in FIG. 2. In someembodiments, the plurality of reference models may include cardiac imagedata of a patient scanned at different times, different locations, anddifferent angles. In some embodiments, the plurality of sets of cardiacdata may include cardiac image data of different patients scanned atdifferent positions and different angles. In some embodiments, theobtaining unit 610 may also be configured to obtain a modelingalgorithm, a parameter, or the like. The obtaining unit 610 may send theobtained plurality of reference models and/or other information to theregistration unit 620, the labelling unit 630, the average modelgeneration unit 640, or the correlation factor generation unit 650.

The registration unit 620 may be configured to adjust the plurality ofreference models obtained by the obtaining unit 610 based on an imageregistration algorithm, and make the positions, ratios, etc. of theplurality of reference models consistent. The image registration mayinclude a spatial dimensional-based registration, a feature-basedregistration, a transformation-based registration, an optimizationalgorithm-based registration, an image modality-based registration, abody-based registration, or the like. In some embodiments, theregistration unit 620 may register the plurality of reference modelsinto the same coordinate system. The registration unit 620 may send theregistered plurality of reference models to the storage module 430 forstorage, or to the labelling unit 630 and/or the average modelgeneration unit 640 for further processing.

The labelling unit 630 may be configured to label a plurality of datapoints (also referred to as point sets) at the edge of the chamber ofthe plurality of reference model. The cardiac image or model may be theplurality of reference models after the image registration by theregistration unit 620, or may be an average model constructed by theaverage model generation unit 640. For example, the chamber edge may bemanually labelled by the user on the plurality of reference models afterthe image registration by the registration unit 620. As another example,the chamber edge may be automatically labelled by the labelling unit 630according to a distinct chamber edge feature. In some embodiments, thelabelling unit 630 may divide the entire cardiac image or model on theplurality of reference models into six parts, such as the leftventricle, the left atrium, the right ventricle, the right atrium, themyocardium, and the aorta. In some embodiments, the labelling unit 630may divide the entire cardiac image or model on the plurality ofreference models into sharp and non-sharp categories according to thechange degree (also referred to as gradient) of the edge of the chamberon the plurality of reference models. Specifically, the labelling unit630 may label the edge points of the chambers that are connected to theoutside or that have small change degree relative to the outside as thesharp category, and label the edge points that are connected to otherinside chambers or that have large change degree relative to the outsideas the non-sharp category, as indicated by two arrows in FIG. 17. Forexample, the labelling unit 630 may divide the entire cardiac image ormodel on the plurality of reference models into 10 categories, such asthe left ventricular margin, the left atrial sharp edge, the left atrialnon-sharp edge, the right ventricular sharp edge, the right ventricularnon-sharp edge, the right atrial sharp margin, the right atrialnon-sharp edge, the aortic margin, the left myocardium sharp edge, andthe left myocardial non-sharp edge (as shown in FIG. 18). In someembodiments, the labelling unit 630 may register the plurality ofreference models into a same coordinate system and the labelling unit630 may label the chamber edges on the plurality of reference models bycomparing the plurality of reference models with the positions of thepoints on the average model obtained by the average model generationunit 640. For example, the labelling unit 630 may determine the categoryof the point on the average model that is closest to the correspondingpoint on the reference model as the category of the point on thereference model. The labelling unit 630 may send the plurality ofreference models with a set of chamber edge points to the storage module430 for storage, or to the training module 450, the average modelgeneration unit 640, and/or the correlation factor generation unit 650for further processing or computing.

The average model generation unit 640 may be configured to construct athree-dimensional cardiac average grid model. In some embodiments, theaverage model generation unit 640 may extract the chamber edges of thelabelled plurality of reference models or the labelled average model.The average model generation unit 640 may also obtain a plurality ofreference grid models by processing the chamber edge models in eachreference model or the average model. The average model generation unit640 may then obtain the average grid model based on an image modelconstruction algorithm. The image model construction algorithm mayinclude a Point Distribution Model (PDM), an Active Shape Model (ASM),an Active Contour Model (also called Snakes), an Active Appearance Model(AAM), or the like. In some embodiments, the average model generationunit 640 may divide the labelled cardiac average model into sixindependent or combined sub-models, for example, the left ventriclemodel, the left atrial model, the right ventricle model, the rightatrial model, the left myocardial model, and the aortic model (as shownin FIG. 20). In some embodiments, the average model generation unit 640may extract the plurality of chamber edges, determine the distributionof control points on the plurality of chamber edges, and form a networkby connecting the control points. In some embodiments, the average modelgeneration unit 640 may obtain the average grid model of the heartchamber and the corresponding model values such as feature values andfeature vectors using an ASM modeling algorithm based on the grid model.In some embodiments, the average model generation unit 640 may considerthe influence of the correlation factor on the control point during theaverage model computation. For example, in the ASM computation, theaverage model generation unit 640 may use a weighted average (i.e.,Σ(F_(i)*W_(i))) to compute the adjustment result of the control point.F_(i) is the deformation parameter of a chamber, and W_(i) is theinfluence coefficient or the weight of the chamber to the control point.The weighted average calculation based on the correlation factor maycause the adjustment of the control point on the model to be affected bythe result of the chamber, thereby achieving the purpose of correlatingthe plurality of chambers. The average model generation unit 640 maysend the three-dimensional cardiac average grid model to the storagemodule 430 for storage or to the correlation factor generation unit 650for computation. The average model generation unit 640 may further sendthe three-dimensional cardiac average grid model to the training module450 and/or the matching module 460 for further processing.

The correlation factor generation unit 650 may be configured toestablish a relationship between each chamber and the control points onthe average grid model. In some embodiments, the relationship may be atwo-dimensional correlation factor matrix (also referred to as matrix)of the chamber and the control point. The value of the matrix mayrepresent the influence coefficient or weight of each chamber on eachcontrol point. In some embodiments, the value of the matrix may be anyreal number between 0-1. For example, the exemplary two-dimensionalcorrelation factor matrix may be shown as:

Left Left Right Right ventricle atrium ventricle atrium Myocardium Aortacontrol 1 0 0 0 0 0 point 1 control 0.8 0.2 0 0 0 0 point 2

In some embodiments, the control point 1 may belong to the leftventricle and may not be in the junction of the left ventricle andatrium. Therefore, the influence coefficient of control point 1 withrespect to the left ventricle is 1 and other chambers is 0. The controlpoint 2 may belong to the left ventricle and may be located at thejunction of the left ventricle and the left atrium. Therefore, theinfluence coefficient of point 2 with respect to the left ventricle is0.8 and the left atrium is 0.2.

In some embodiments, the correlation factor generation unit 650 mayestablish a correlation factor matrix according to the chamberattribution of the control points on the grid model and the positionalrelationships between the control points and other chambers. In someembodiments, the correlation factor generation unit 650 may compute aninfluence range or influence coefficient of the correlation factoraccording to the distance between the control point and other chambers.For example, the correlation factor generation unit 650 may control thecomputation of the correlation factor by the maximum distance of thecontrol point distance from other chambers. In some embodiments, thecorrelation factor generation unit 650 may adjust the influence rangeand the influence coefficient between different chambers according tothe compactness degree between the chambers. As shown in FIG. 21, in thegrid control point model, a control point with a light color mayindicate that the control point is only affected by the chamber, and thedark chamber junction may indicate that the control points therein areaffected by the plurality of connected chambers. The darker the color,the greater the influence by other chambers. The correlation factorgeneration unit 650 may send the obtained two-dimensional correlationfactor matrix to the storage module 430 for storage, or to the averagemodel generation unit 640 and/or the adjustment module 470 for weightingcomputation.

It should be noted that the above descriptions of the model constructionmodule 440 are merely for convenience of illustration and are notintended to limit the scope of the present disclosure. It will beunderstood that those skilled in the art, after understanding theworking principle of the module, it is possible to make any combinationof the units in the module, or to form a subsystem connected with otherunits, and make various modifications and changes to the form anddetails of the module without departing from the principle. For example,the registration unit 620 and/or the labelling unit 630 may be omitted,or may be combined with the obtaining unit 610 and the storage module430. As another example, the plurality of reference models or theaverage model may include cardiac data or models that have beenedge-labelled by the user. As a further example, the plurality ofreference models or the average model may include cardiac data withrough segmented chambers or precise segmented chambers. All suchvariations are within the protection scope of the present disclosure.

FIG. 7 is a flowchart illustrating an exemplary process for constructingan average model according to some embodiments of the presentdisclosure. In 710, a plurality of cardiac reference models may beobtained. The plurality of cardiac reference models may be obtained fromthe database 120, the user input, or storage devices outside the controland processing system 100. In some embodiments, the plurality of cardiacreference models may include cardiac image data of a patient scanned atdifferent times, different locations, and different angles. In someembodiments, the plurality of cardiac reference models may includecardiac image data of different patients scanned at different positionsand different angles. In some embodiments, the plurality of cardiacreference models may include cardiac data or models that have beenedge-labelled by experts. In some embodiments, the plurality ofreference models may include cardiac data with rough segmented chambersor precise segmented chambers.

In 720, an image registration may be performed on the obtained pluralityof reference models. The operation 720 may be implemented by theregistration unit 620 of the model construction module 440. In someembodiments, any two reference models may be transformed into the samecoordinate system using one or more operations (such as translation,rotation, scaling, etc.), making the points corresponding to the sameposition in space in the two reference models being one-to-onecorrespondence, thereby implementing the information fuse. In someembodiments, the image registration may include a spatialdimensional-based registration, a feature-based registration, atransformation-based registration, an optimization algorithm-basedregistration, an image modality-based registration, a body-basedregistration, or the like. The spatial dimension-based registration mayinclude a 2D/2D registration, a 2D/3D registration, a 3D/3Dregistration, or the like. The feature-based registration may include aregistration based on feature points (e.g., discontinuity points, shapeturning points, line intersections, etc.), a registration based on faceregions (e.g., curves, surfaces, etc.), a pixel-based registration, anexternal feature-based registration, or the like. Thetransformation-based registration may include a stiffnesstransformation-based registration, an affine transformation-basedregistration, a projection transformation-based registration, a curvetransformation-based registration, or the like. The optimizationalgorithm-based registration may include a registration based ongradient descent algorithm, a registration based on a Newton algorithm,a registration based on a Powell algorithm, a registration based on agenetic algorithm, or the like. The image modality-based registrationmay include a single modality-based registration and/or amultimodality-based registration. The body-based registration mayinclude a registration based on images from a same patient, aregistration based on images from different patients, and/or aregistration based on patient data and atlas.

In 730, an edge of the chamber may be labelled on the plurality ofregistered reference models. The operation 730 may be implemented by thelabelling unit 630 of the model construction module 440. In someembodiments, the edge points may be manually labelled by the user on theplurality of heart reference models. The edge points on each referencemodel may divide the heart into six parts: the left ventricle, the leftatrium, the right ventricle, the right atrium, the myocardium, and theaorta, respectively. In some embodiments, the heart may be divided into10 categories according to the change degree of the edge of the chamberrelative to the outside and inside, that is, the left ventricularmargin, the left atrial sharp edge, the left atrial non-sharp edge, theright ventricular sharp edge, the right ventricular non-sharp edge, theright atrial sharp margin, the right atrial non-sharp edge, the aorticmargin, the left myocardium sharp edge, and the left myocardialnon-sharp edge (as shown in FIG. 18). The sharp edge may refer to thatthe edge of the chamber is connected to the outside or does not havesignificant changes. The non-sharp may refer to that the edge of thechamber is connected to the inside or other chambers or has significantchanges.

In 740, control points on the plurality of reference models may bedetermined. The operation 740 may be implemented by the average modelgeneration unit 640 of the model construction module 440 based on theplurality of reference models after the image registration and thechamber edge labelling. In some embodiments, the axis of each chambermay be determined based on the image registration results and thechamber edge labelling information of the plurality of reference models.The axis may be the direction of the line connecting any two designatedpoints on the chamber. For example, the axis may be a long axis formedby a line connecting the two points of the farthest apart in thechamber. In some embodiments, the labelled chamber edges of theplurality of reference models may be extracted separately, each chambermay be sliced along the cross-section direction of the determined axison each chamber, and a dense point set may be formed at the edge of theslice based on the cross-section and the curved surface features todetermine a point model of the average model (as shown in FIG. 19). Insome embodiments, the control points on each chamber may be determinedaccording to the point model. The control points may be a subset of thepoint set on the point model. For example, if the subset is larger, thegrid model may be larger, the computation amount during the heartsegmentation process may be greater, and the segmentation result may bebetter. If the selected subset is smaller, the grid model may besmaller, the computation amount during the heart segmentation processmay be smaller, and the segmentation speed may be faster. In someembodiments, the number of control points on the chamber may be varied.For example, in the rough segmentation phase, the number of controlpoints may be less so that the edge of the chamber can be quicklypositioned. In the precise segmentation phase, the number of controlpoints may be larger, thereby achieving the precise segmentation of theedge of the chamber.

In 750, a heart average grid model may be constructed based on thecontrol points. In some embodiments, in 750, different points may beconnected to form a polygonal network based on the relationships betweenthe control points. For example, in some embodiments, a triangle networkmay be formed by connecting the adjacent control points on adjacentslices. In some embodiments, the average grid model may be obtained byan image deformation algorithm. The image deformation algorithm mayinclude a Point Distribution Model (PDM), an Active Shape Model (ASM),an Active Contour Model (also called Snakes), an Active Appearance Model(AAM), or the like. For example, in some embodiments, the average gridmodel of the plurality of cardiac reference models may be obtained basedon a triangular network constructed by the control points using the ASMcomputation algorithm (as shown in FIG. 20). In some embodiments, theoperation 750 may include performing a weighted average modelcomputation on the control point grid model based on a two-dimensionalcorrelation factor matrix. For example, in the ASM computation, theaverage model generation unit 640 may perform a weighted average (i.e.,ΣF_(i)*W_(i)) to determine an adjustment result of the control point.F_(i) is the deformation parameter of a chamber, and W_(i) is theinfluence coefficient or weight value of the chamber to the controlpoint.

It should be noted that the above descriptions of constructing theaverage model by the model construction module 440are merely forconvenience of illustration and are not intended to limit the scope ofthe present disclosure. It should be understood that, for those skilledin the art, after understanding the working principle of the module, itis possible to arbitrarily adjust the order of each operation, or addand remove some operations without depart from the principle. Forexample, the operation 710 and the operation 720 may be combined. Asanother example, operations 730-750 may be repeated multiple times. Allsuch variations are within the protection scope of the presentdisclosure.

FIG. 8 is a schematic diagram illustrating an exemplary training moduleaccording to some embodiments of the present disclosure. The trainingmodule 450 may include a classification unit 810 and a classifiergeneration unit 820. The connection between modules within the trainingmodule 440 may be a wired connection, a wireless connection, or acombination thereof. Any module may be local, remote, or a combinationthereof.

The classification unit 810 may be configured to divide the possiblechamber edge points on the plurality of reference models or the averagemodel into different chamber categories. The functions may beimplemented by the processor 220. In some embodiments, theclassification unit 810 may classify the possible edge points on thereference model or the average model according to the chamber categorydivided by the labelling unit 630 (as shown in FIG. 22). For example,the classification unit 810 may divide the possible edge points near thechamber on the reference model or the average model into ten chambercategories, respectively, that is, the left ventricular margin, thesharp left atrial margin, the left atrial non-sharp edge, the rightventricular sharp margin, the right ventricular non-sharp edge, theright atrial sharp edge, the right atrial non-sharp edge, the aorticmargin, the left myocardium sharp edge, and the left myocardialnon-sharp edge. The classification may be implemented by variousclassification algorithms, including but not limited to a Decision Treeclassification algorithm, a Bayes classification algorithm, anartificial neural network (ANN) classification algorithm, a K-proximity(kNN), support vector machine (SVM), a classification algorithm based onassociation rules, an integrated learning classification algorithm, orthe like. In some embodiments, the classification unit 810 may dividepoints near the edge of the chamber into positive samples and negativesamples based on the distance between the points and the edge of thechamber. For example, the positive samples may be data points within acertain threshold range from the edge of the chamber, and the negativesamples may be data points that are farther away from the edge of thechamber and/or other random positions in the space. In some embodiments,the classification unit 810 may send the classification results or dataof the possible edge points of the reference model or the average modelto the storage module 430 for storage, or to the classifier generationunit 820 for further processing.

The classifier generation unit 820 may be configured to obtain a trainedclassifier. In some embodiments, the classifier generation unit 820 mayperform classifier training on the edge points of the reference model orthe average model according to the edge point categories divided by theclassification unit 810, to obtain a trained classifier (as shown inFIG. 23). In some embodiments, the classifier generation unit 820 mayutilize a PBT training classifier. In some embodiments, after receivinga coordinate point, the trained classifier may output a probability ofthe coordinate point. The probability may refer to a probability that apoint is at the edge of the chamber. In some embodiments, the classifiergeneration unit 820 may send the trained classifier to the storagemodule 430 for storage, or to the matching module 460 and/or theadjustment module 470 for computation.

It should be noted that the above descriptions of the training module450 are for convenience of illustration and are not intended to limitthe scope of the present disclosure. It will be understood that thoseskilled in the art, after understanding the working principle of themodule, it is possible to make any combination of the units within themodule, or to form a subsystem connected with other units, and makevarious modifications and changes to the form and details ofimplementing the module without departing from the principle. Forexample, the classification unit 810 may perform a chamber division onthe plurality of reference models or the average model such that thedivided chamber categories are more precise with respect to the labelledchamber categories. All such variations are within the protection scopeof the present disclosure.

FIG. 9 is a flowchart illustrating an exemplary process for training aclassifier according to some embodiments of the present disclosure. In910, the classification unit 810 of the training module 450 may obtainsample points from the plurality of reference models or the averagemodel. In some embodiments, the training module 450 may extract the edgeof the chamber based on the chamber segmentation results on theplurality of reference models or the average model after labelled (asshown in FIG. 22), and take the points within a certain range near theedge of each chamber as positive samples and the points far away fromthe edge of the chamber and other random positions in the space asnegative samples. For example, the range of the edge of the chamber maybe 0.1 cm, 0.5 cm, 1 cm, 2 cm, etc.

In 920, the classification unit 810 of the training module 450 mayclassify the obtained positive sample points and negative sample points.In some embodiments, the training module 450 may classify the positivesample points and the negative sample points into different chambercategories according to a classification algorithm. In some embodiments,the positive samples may be the points within a certain range of theedge of the average model, and the negative samples may be the pointsoutside the certain range of the edge of the average model. In someembodiments, the certain range of the edge of the average model may beset to zero, and the positive samples may be the edge points of theaverage model. In some embodiments, the positive samples may beclassified based on the sharpness and the location of the sample points.In some embodiments, the location of the sample points may refer to thechambers of the positive samples and the negative samples. For example,the training module 450 may divide the positive sample points and thenegative sample points into ten chamber categories according to thelabelled chamber categories, that is, the left ventricular margin, thesharp left atrial margin, the left atrial non-sharp edge, the rightventricular sharp margin, the right ventricular non-sharp edge, theright atrial sharp edge, the right atrial non-sharp edge, the aorticmargin, the left myocardium sharp edge, and the left myocardialnon-sharp edge. The classification method may include a Decision Treeclassification algorithm, a Bayes classification algorithm, anartificial neural network (ANN) classification algorithm, a k-proximity(kNN), a support vector machine (SVM), a classification algorithm basedon association rules, an integrated learning classification algorithm,etc. The Decision Tree classification algorithm may include ID3, C4.5,C5.0, CART, PUBLIC, SLIQ, a SPRINT algorithm, or the like. The Bayesianclassification algorithm may include a naive Bayesian algorithm, a TANalgorithm (tree augmented Bayes network), or the like. The artificialneural network classification algorithm may include a back propagation(BP) network, a radial basis RBF network, a Hopfield network, a randomneural network (for example, Boltzmann machine), a competitive neuralnetwork (for example, Hamming network, self-organizing map network,etc.), or the like. The classification algorithm based on associationrules may include CBA, ADT, CMAR, or the like. The integrated learningclassification algorithms may include Bagging, Boosting, AdpBoosting,PBT, etc.

In 930, the training module 450 may obtain the trained classifier. Insome embodiments, the classifier generation unit 820 of the trainingmodule 450 may train the sample point categories through the PBTalgorithm and obtain one or more trained classifiers (as shown in FIG.23). The PBT may include a two-level PBT algorithm or a multi-level PBTalgorithm. In some embodiments, the classifier may include one or moreclassifiers (also referred to as “first classifiers”) trained by takingthe points within a certain range of the edge of the plurality ofreference models or the average model as the positive samples. In someembodiments, the classifier may include one or more classifiers (alsoreferred to as “second classifiers”) trained by taking the points withina certain range of the edge of the image to be processed as the positivesamples.

It should be noted that the above descriptions of the process oftraining the classifier by the training module 450 are merely forconvenience of illustration and are not intended to limit the scope ofthe present disclosure. It should be understood that, for those skilledin the art, after understanding the working principle of the module, itis possible to arbitrarily adjust the order of each operation, or addand remove some operations without departing from the principle. Forexample, the operation 910 and the operation 920 may not distinguish thepositive samples and the negative samples, and directly classify all thepoints near the edge of the chamber. As another example, the maximumdistance between the edge of the chamber and the positive sample pointsand/or the negative sample points may be 2 cm. All such variations arewithin the protection scope of the present disclosure.

FIG. 10 is a schematic diagram illustrating an exemplary model matchingmodule according to some embodiments of the present disclosure. As shownin FIG.10, the matching module 460 may include an obtaining unit 1010,an image point extraction unit 1020, and a model matching unit 1030. Theconnection between the units of the matching module 460 may be a wiredconnection, a wireless connection, or a combination thereof. Any unitmay be local, remote, or a combination thereof.

The obtaining unit 1010 may obtain an image. The obtained image may bean image to be processed. In some embodiments, the image may be areconstructed image based on image data. The reconstructed image may beobtained from other modules of the processing device 130. For example,the reconstructed image may be obtained by the obtaining unit 1010 fromthe image reconstruction module 420. As another example, thereconstructed image may be stored in the storage module 430 after beingreconstructed by the image reconstruction module 420. The reconstructedimage may be obtained from the storage module 430. In some embodiments,the image may be an image inputted to the system via an external device.For example, the external device may input the image into the system viathe communication port 250. In some embodiments, the obtaining unit 1010may obtain an average model. The average model may be athree-dimensional cardiac average grid model generated by the averagemodel generation unit 640. In some embodiments, the obtaining unit 1010may obtain the first classifier trained by the training module 450. Thefirst classifier may be obtained based on a point classifier. The imagefeature may be related to sharpness and location.

In some embodiments, the obtaining unit 1010 may obtain parametersrequired when the model matching module 460 performs an image matching.For example, the obtaining unit 1010 may obtain parameters for theGeneralized Hough Transform. In some embodiments, the parameters of theGeneralized Hough Transform may be obtained based on thethree-dimensional average grid model and the control points of itschamber edge. For example, the offset vector (hereinafter referred to asa gradient vector) corresponding to the control points of each gradientdirection may be obtained by determining the centroid of the edge of theaverage model, and computing the offset of all control points on theedge of the average model relative to the centroid and the gradientdirection relative to the centroid. In some embodiments, the averagemodel may be placed in an x-y-z coordinate system, and the coordinatesof each gradient vector in the x-y-z coordinate system may bedetermined. In some embodiments, the coordinates of each gradient vectormay be converted to coordinates in a polar coordinate system.Specifically, an intersection angle of the projection of the gradientvector in the x-y plane and the x coordinate axis may be taken as afirst angle θ, ranging from −180 degrees to 180 degrees. An intersectionangle of the gradient vector and the x-y plane may be taken as a secondangle φ, ranging from −90 degrees to 90 degrees. In some embodiments,the two angles θ and φ representing the gradient vector may bediscretized to obtain a table (also referred to as R-table) as describedbelow. In some embodiments, the offset on the R-table may be scaled orrotated with different angles to examine the shapes of different sizesor different angles.

angles of the gradient the offset of the φ, θ corresponding points 0, 90(x0, y0, z0), (x3, y3, z3), . . . 0, 80 (x2, y2, z2), (x5, y5, z5), . .. . . . . . . 10, 90  (x4, y4, z4), (x6, y6, z6), . . . . . . . . .

The image point extraction unit 1020 may obtain an edge probability mapof the image to be processed. Specifically, in some embodiments, theimage point extraction unit 1020 may compute a probability that eachpoint on the image to be processed is a chamber edge by inputting thecoordinates of the points on the image to be processed into theclassifier obtained by the obtaining unit 1010. Thr image pointextraction unit 1020 may then obtain an edge probability map of theimage to be processed according to a probability distribution of eachpoint. In some embodiments, the edge probability map may include agrayscale gradient map, a color gradient map (as shown in FIG. 24), orthe like. In some embodiments, the image point extraction unit 1020 maydetermine the point(s) on the edge probability map whose probabilityvalue is greater than a certain threshold as first edge point(s). Thethreshold may be any real number between 0-1, for example, 0.3, 0.5, orthe like.

The model matching unit 1030 may match the average model to the image tobe processed. In some embodiments, the model matching unit 1030 maymatch the average model to the edge probability map of the image to beprocessed by the weighted Generalized Hough Transform. The weightedGeneralized Hough Transform may include obtaining all possible edgereference points on the image to be processed according to the firstedge point and the R-table, determining a probability accumulation valueof all the edge reference points using a weighted accumulationalgorithm, and determining the edge reference point with the largestprobability accumulation value as the centroid of the image. Thetransformation parameter of the model centroid to the image centroid maybe used as the transformation parameter of the model. The edge referencepoints may be obtained by performing a coordinate transformation on thefirst edge point of the image to be processed according to theparameters in the R-table. The weighted accumulation may be a process ofaccumulating the first edge point probabilities corresponding to thesame edge reference points (referring to the behavior that the firstedge points fall to the same edge reference points after the parametersof the R-table are offset). According to the obtained image centroid,the model centroid may be transformed to a position coincident with theimage centroid based on the transformation parameter. The transformationparameter may include a rotation angle, a scaling ratio, or the like. Insome embodiments, the model matching unit 1030 may perform an operation(such as rotate, scale, or the like) on the points of the modelaccording to the determined transformation parameter, therebydetermining a model matched with the image to be processed (as shown inFIG. 25).

It should be noted that the above descriptions of the matching module460 are merely for convenience of illustration and are not intended tolimit the scope of the present disclosure. It will be understood thatthose skilled in the art, after understanding the working principle ofthe module, it is possible to make any combination of the units withinthe module, or to form a subsystem connected with other units, and makevarious modifications and changes to the form and details ofimplementing the module without departing from the principle. Forexample, the image point extraction unit 1020 may be removed, and theedge probability map of the image to be processed may be determined bythe training module 450. All such variations are within the protectionscope of the present disclosure.

FIG. 11 is a flowchart illustrating an exemplary process for matching anaverage model and a reconstructed image according to some embodiments ofthe present disclosure. In 1110, an average model, an image to beprocessed, and a trained second classifier may be obtained. In someembodiments, the average model may be a three-dimensional cardiacaverage grid model obtained by the average model generation unit 640based on a plurality of reference models using an image modelconstruction algorithm. The image model construction algorithm mayinclude a Point Distribution Model (PDM), an Active Shape Model (ASM),an Active Contour Model (also called Snakes), an Active Appearance Model(AAM), or the like. The operation 1110 may be implemented by theobtaining unit 1010. In some embodiments, the image to be processed maybe an image reconstructed by the image reconstruction module 420. Insome embodiments, the operation 1110 may obtain an R-table based on theaverage model.

In 1120, a parameter of a Generalized Hough Transform may be determined.In some embodiments, the operation 1120 may include obtaining first edgepoints of the image to be processed based on an edge probability map ofthe image to be processed. The first edge points may be points on theedge probability map of the image to be processed having a probabilitygreater than a certain threshold, for example, the probability may be0.3. In some embodiments, the edge probability map may be obtained byinputting the coordinates of the points on the image to be processedinto the classifier obtained by the obtaining unit 1010, computing aprobability that the points on the image to be processed are on the edgeof the chamber, and determining a probability distribution of eachpoint. In some embodiments, the angles θ and φ corresponding to thegradient direction of the first edge points on the image to be processedmay be computed. The offset of the first edge points may be determinedaccording to the R-table. The difference between the coordinate value ofthe first edge points and all the corresponding offsets may be used asthe coordinate values of all the possible edge reference points. In someembodiments,a weighted accumulation may be performed on all the edgereference points according to the number of voting the edge referencepoints and the probability value corresponding to the first edge points.The weighted accumulation may be an accumulation of the probabilities ofthe first edge points corresponding to the same edge reference points.In some embodiments, the parameter in the R-table corresponding to theedge reference point with the largest probability accumulation value maybe used as a transformation parameter of the image to be processed. Thetransformation parameter may include a rotation angle, a scaling ratio,or the like. The weighted accumulation algorithm may be representedaccording to Equation (1):

$\begin{matrix}{{S\left( {\theta,\phi,r} \right)} = {\arg \; {\max\limits_{j}{\sum\limits_{i}{p_{i}\sigma}}}}} & (1)\end{matrix}$

Wherein, i is the index of the first edge point; j is the index of thepossible edge reference point voted on the voting image; p is theprobability value of each first edge point; and σ is a 0, 1 binaryfunction. When the i-th first edge point has a voting contribution atthe j-th possible edge reference point, the value is 1. Alternatively,if the i-th first edge point does not have a voting contribution at thej-th possible edge reference point, the value is 0.

In 1130, the model corresponding to the image to be processed may beobtained. In some embodiments, the first edge point on the image to beprocessed may be transformed based on the determined weightedGeneralized Hough Transform parameter. For example, according to theangle and scaling ratio in the R-table corresponding to the edgereference points, the coordinates of the first edge point on the imageto be processed may be transformed, and the information on the averagemodel may be mapped to the image to be processed to obtain the image tobe processed corresponding to the average grid model.

FIG. 12 is a schematic diagram illustrating an exemplary adjustmentmodule according to some embodiments of the present disclosure. As shownin FIG. 12, the adjustment module 470 may include an obtaining unit1210, an object point determination unit 1220, and a modeltransformation unit 1230. The connection between the units of theadjustment module 470 may be a wired connection, a wireless connection,or a combination thereof. Any unit may be local, remote, or acombination thereof.

The obtaining unit 1210 may obtain the model and the trained secondclassifier. Specifically, the obtaining unit 1210 may obtain coordinatedata of second edge points on the model. In some embodiments, the secondedge points of the model may be the control points on the model. In someembodiments, the obtaining unit 1210 may obtain the second classifiertrained by the training module 450. The classifier may include 10classifiers trained using the PBT classification algorithm based on tenchamber categories (e.g., the left ventricular edge, the sharp edge ofthe left atrium, the non-sharp edge of the left atrium, the sharp edgeof the right ventricle, the non-sharp edge of the right ventricle, thesharp edge of the right atrium, the non-sharp edge of the right atrium,the edge of the aorta, the sharp edge of the left myocardium, and thenon-sharp edge of the left myocardium) divided by chamber and edgesharpness. In some embodiments, the grayscale changes on the inner andouter sides of a chamber edge may not be obvious and the sharpnessdegree may be low. Therefore, the chamber edge may not be classifiedaccording to the sharpness degree. In some embodiments, the obtainingunit 1210 may obtain the model processed by the model transformationunit 1230.

The object point determination unit 1220 may determine the object pointscorresponding to the second edge points on the model. Taking a secondedge point on the model as an example, the object point determinationunit 1220 may determine a plurality of candidate points around thesecond edge point of the model. In some embodiments, the object pointdetermination unit 1220 may input the determined plurality of candidatepoints around the second edge point of the model into the classifierobtained by the obtaining unit 1210. The object point determination unit1220 may determine probabilities of the second edge point of the modeland the plurality of candidate points corresponding to the edge of theimage. The object point determination unit 1220 may determine an objectpoint of the second edge point of the model based on the probabilities.In some embodiments, the object point determination unit 1220 maydetermine the object points for all the second edge points on the model.

The model transformation unit 1230 may adjust the model. In someembodiments, the model transformation unit 1230 may adjust the positionof the model edge point(s) based on the object point(s) determined bythe object point determination unit 1220. The adjustment may include asimilarity transformation, a segmentation affine transformation, anenergy function based micro-variation, or the like. In some embodiments,the model transformation unit 1230 may repeatedly adjust the model. Eachadjustment may require re-determining the object point. In someembodiments, the model transformation unit 1230 may determine whether apredetermined condition is satisfied after the adjustment. For example,the predetermined conditions may include whether the number of modeladjustments reaches a certain threshold. If the number of modeladjustments reaches the certain threshold, a precisely matched model maybe output. If the number of model adjustments is less than the certainthreshold, a signal may be sent to the object point determination unit1220, the determination of the object point may be performed again, andthe model edge point may be transformed again by the modeltransformation unit 1230. In some embodiments, the model transformationunit 1230 may obtain a precisely adjusted heart chamber model. Theprecisely adjusted heart chamber model may be very close to the realheart.

It should be noted that the descriptions of the adjustment module 470are merely for convenience of illustration and are not intended to limitthe scope of the present disclosure. It will be understood that thoseskilled in the art, after understanding the working principle of themodule, it is possible to make any combination of the units of themodule, or to form a subsystem connected with other units, and makevarious modifications and changes to the form and details ofimplementing the module without departing from the principle. Forexample, the model transformation unit 1230 may preset the number ofcycles instead of determining the number of cycles of the adjustmentmodule 470 based on the threshold determination. All such variations arewithin the protection scope of the present disclosure.

FIG. 13 is a flowchart illustrating an exemplary process for adjusting amodel according to some embodiments of the present disclosure. In 1310,the second edge points on the model and the trained classifier may beobtained. In some embodiments, the classifiers obtained by the obtainingunit 1210 and by the obtaining unit 1010 may not be the same type. Theclassifier obtained by the obtaining unit 1010 may be trained by takingthe points within a certain range of the edge of the average grid modelas the positive samples by the training module 450. The classifierobtained by the obtaining unit 1210 may be trained by taking the pointswithin a certain range of the edge of the image to be processed as thepositive samples. In some embodiments, the classifier obtained by theobtaining unit 1010 may be a first classifier, and the classifierobtained by the obtaining unit 1210 may be a second classifier.

In 1320, object points of the second edge points on the model may bedetermined based on the second classifier. In some embodiments, theoperation 1320 may include inputting candidate points within a certainrange of the second edge points of a model into the second classifier,and obtain a probability that the candidate points within the certainrange of the second edge points of the model belongs to the edge of theimage. In some embodiments, the object point determination unit 1220 maydetermine the object points of the second edge points of the model basedon the determined probabilities. In some embodiments, the second edgepoints on the model may be the inner edge points of the model (the inneredge of the second edge), corresponding to the inner edge of the firstedge of the cardiac image data. The process of transforming the secondedge points to the object points may be a process of precisely matchingthe inner edge of the chamber on the model with the inner edge of thefirst edge of the cardiac image data. The inner edge may refer to anedge for precise matching. When the process disclosed in the presentdisclosure is used for other objects, organs or tissues, the inner edgemay not be necessarily geometrically internal and not be necessarilyinside the outer edge.

In 1330, the second edge points on the model may be transformed into theobject points based on the determined object points. In someembodiments, the operation 1330 may transform the second edge points onthe model using multiple transformation algorithms. For example, themodel transformation unit 1230 may apply the similarity change and theaffine transformation to correct the second edge points on the model.

In 1340, whether the adjustment result satisfies the predeterminedcondition may be determined. In some embodiments, the predeterminedcondition may be whether the number of adjustments reaches a certainthreshold. In some embodiments, the threshold may be adjustable. If thenumber of adjustments reaches the certain threshold, the process mayproceed to operation 1350, and the precisely matched model may beoutputted. If the number of adjustments is less than the certainthreshold, the process may return to operation 1320, and the objectpoint corresponding to the new model edge point may be determined by theobject point determination unit 1220 based on the new model edge point.

FIG. 14 is a flowchart illustrating an exemplary process for determiningan object point according to some embodiments of the present disclosure.The process 1400 may be implemented by the object point determinationunit 1220. FIG. 14 is a process for determining an object point of anedge point on the average model, but those skilled in the art shouldunderstand that the process may be used to determine a plurality ofobject points corresponding to a plurality of edge points. In someembodiments, the process 1400 may be an embodiment of operation 1320.

In 1410, a normal of an average model edge point may be determined. Insome embodiments, the direction of the normal may be directed from theinside of the average model to the outside of the average model. Thespecific normal acquisition process may be found elsewhere in thepresent disclosure, for example, process 1500 and the descriptionthereof.

In 1420, the step size and the search range along the direction of thenormal of the average model edge point may be obtained. In someembodiments, the step size and the search range may be predeterminedvalues. In some embodiments, the step size and the search range may beinputted by a user. For example, the user may input the step size andthe search range into the processing device 130 via the communicationport 250 using an external device. In some embodiments, the search rangemay be a line segment starting from the edge point of the model andalong at least one of two directions (toward the outer side or the innerside of the model) of the normal.

In 1430, one or more candidate points may be determined based on thestep size and the search range. For example, if the search range is 10cm and the step size is set to 1 cm, 10 points in both directions alongthe normal may be determined, that is, a total of 21 candidate points(including the edge point) may be determined. In some embodiments, thestep size and the number of steps may be determined, and the candidatepoints may be determined based on the step size and the number of steps.For example, if the step size is set to 0.5 cm and the number of stepsis set to 3, 3 points in both directions along the normal may bedetermined and the farthest candidate point distant from the edge pointmay be 1.5 cm, i.e., a total of 7 candidate points.

In 1440, probabilities that the one or more candidate points correspondto a range of the image edges may be determined. In some embodiments,the second classifier may be trained by taking the points within acertain range of the image edge as the positive samples. The certainrange may be a predetermined value set by the machine or the user. Forexample, the predetermined value may be 1 cm.

In 1450, one of the one or more candidate points may be determined as anobject point based on the probabilities that the one or more candidatepoints correspond to the certain range of the image edges. In someembodiments, the object point may be obtained according to Equation (2):

F _(i)=max(P _(i) −λ*d _(i) ²)   (2)

Wherein, P_(i) is the probability that the candidate point correspondsto a certain range of the image edge; d_(i) is the European distance ofthe candidate point and the edge point of the average model; λ is theweight, which is a constant used to balance the relationship between thedistance and the probability.

In some embodiments, a plurality of object points of the plurality ofmodel edge points may be determined based on the process 1400, and thenthe plurality of model edge points and the model may be transformedaccording to the plurality of object points. The specific transformationprocess can be seen, for example, FIG. 16 and description thereof.

FIG. 15 is a flowchart illustrating an exemplary process for determininga normal of an edge point according to some embodiments of the presentdisclosure. In some embodiments, the process 1500 may be an embodimentof operation 1420.

In 1510, a plurality of polygons may be determined based on a pluralityof edge points of the average model. In some embodiments, the pluralityof polygons may be formed by connecting the plurality of edge points.The plurality of polygons may have the shape of a triangle, aquadrangle, a polygon, or the like. In some embodiments, the process ofdetermining the plurality of polygons according to the plurality of edgepoints may also be referred to as a gridization process. The pluralityof polygons may be referred to as grids, and the plurality of edgepoints may be referred to as nodes. In some embodiments, the averagemodel surface may have formed the plurality of polygons corresponding tothe average model edge points, and operation 1510 may be omitted.

In 1520, a plurality of polygons adjacent to an average model edge pointmay be determined.

In 1530, a plurality of normals corresponding to planes of the pluralityof polygons may be determined. In some embodiments, the directions ofthe plurality of normals corresponding to planes of the plurality ofpolygons may be located at the same side (outside or inside the averagemodel). In some embodiments, the plurality of normal vectorscorresponding to the planes of the plurality of polygons may be unitvectors.

In 1540, the normal of the edge point may be determined based on theplurality of normals. In some embodiments, the plurality of normalvectors corresponding to the plurality of polygons may be added oraveraged.

FIG. 16 is a flowchart illustrating an exemplary process fortransforming average model edge points according to some embodiments ofthe present disclosure. In some embodiments, the process 1600 may beimplemented by the model transformation unit 1230.

In 1610, a similarity transformation may be performed on the averagemodel edge points. For example, a grid composed of the average modeledge points may be taken as a whole, and the average model may betransformed (e.g., including translation, rotation, scaling, or thelike) according to the direction of the object point determined by theedge points of the chamber.

In 1620, a segmentation affine transformation may be performed on theaverage model edge points. In some embodiments, the grid composed of theaverage model edge points may be divided according to a certain rule.For example, the heart model may be divided according to the heartchambers. As shown in FIG. 24, the model grid may be divided into sixparts according to the chambers, that is, the left ventricle, the leftatrium, the right ventricle, the right atrium, the aorta, and the leftmyocardium. In some embodiments, the segmentation affine transformationmay refer to the affine transformation of the grids of the dividedparts. The affine transformation may refer to performing a motiontransformation and a shape transformation on a plurality of nodes ofeach part. In some embodiments, the average model edge points may beaffected by the plurality of chambers. The influence of differentchambers on the average model edge points may be expressed in the formof a correlation factor. When an affine transformation is performed, theaverage model edge point may be transformed toward the object point.During the transformation process, the average model edge points may beaffected by the plurality of chambers. The correlation factor may becomethe weight value of the transformation parameter (such as a mobiledisplacement, a deformation ratio, etc.). According to the object pointscorresponding to the edge points and the correlation factor, the modeltransformation unit 1230 may transform the edge points on the averagemodel multi-segment grid to the corresponding positions by performing asegmentation affine transformation.

In 1630, an energy function based micro-variation may be performed onthe average model edge points. In some embodiments, the energy functionmay be expressed as:

E=E _(ext)+Σ_(c)α_(c) *E _(int) ^(c)   (3)

Wherein, E_(ext) is an external energy, indicating a relationshipbetween the current point and the object point; E_(int) is an internalenergy, indicating a relationship between the current point and an edgepoint of the average model; α is the weight, used to balance theinternal and external energies, different chambers having differentweights; c denotes each chamber. If the current point is close to theobject point and close to an edge point of the average model, the energyfunction may be minimum, that is, the optimal coordinate point may beobtained. The smaller the total energy E is, the more precise the resultmay be.

The external energy function may be expressed as:

$\begin{matrix}{E_{ext} = {\sum\limits_{i}{w_{i}\left( {\frac{\nabla{I\left( v_{i}^{t} \right)}}{{\nabla{I\left( v_{i}^{t} \right)}}}*\left( {v_{i}^{t} - v_{i}} \right)} \right)}^{2}}} & (4)\end{matrix}$

Wherein, i refers to each point; w_(i) refers to the weight of eachpoint (i.e., the reliability of the point); v_(i) is the coordinate ofthe current point, v_(i) ^(t) is the point examined by the PBTclassifier, ∇I(v_(i) ^(t)) is the gradient (or vactor) of the point, and∥∇I(v_(i) ^(t))∥ is the numeral value of the gradient.

The internal energy function may be expressed as:

E _(int)=Σ_(i)Σ_(j)Σ_(k) w _(i,k)((v _(i) −v _(j))−T _(af fine,k)(m _(i)−m _(j)))²   (5)

Wherein, i refers to each point; j is the neighborhood of point i(v_(i)−v_(j) corresponding to the edges of each triangle at the currentpoint position); w_(i,k) is the correlation factor (i.e., the factor ofeach chamber k to the current point i); m_(i), m_(j) are the point onthe average model (determined by PDM/ASM); m_(i)−m_(j) corresponds tothe edges of each triangle of the mesh average model, T_(af fine,k) isthe transformation relationship obtained by the affine transformationPAT of each chamber k. The point coordinates v_(i) arethree-dimensional.

After the weighted Generalized Hough Transform, the model adjustment,and the model transformation, the precisely matched models and imagesmay be obtained. As shown in FIG. 25, the heart chambers of theprecisely matched model may be segmented out clearly and precisely.

Having thus described the basic concepts, it may be rather apparent tothose skilled in the art after reading this detailed disclosure that theforegoing detailed disclosure is intended to be presented by way ofexample only and is not limiting. Various alterations, improvements, andmodifications may occur and are intended to those skilled in the art,though not expressly stated herein. These alterations, improvements, andmodifications are intended to be suggested by this disclosure, and arewithin the spirit and scope of the exemplary embodiments of thisdisclosure.

Moreover, certain terminology has been used to describe embodiments ofthe present disclosure. For example, the terms “one embodiment,” “anembodiment,” and/or “some embodiments” mean that a particular feature,structure or characteristic described in connection with the embodimentis included in at least one embodiment of the present disclosure.Therefore, it is emphasized and should be appreciated that two or morereferences to “an embodiment,” “one embodiment,” or “an alternativeembodiment” in various portions of this specification are notnecessarily all referring to the same embodiment. Furthermore, theparticular features, structures or characteristics may be combined assuitable in one or more embodiments of the present disclosure

Moreover, those skilled in the art will appreciate that the aspects ofthe present disclosure may be illustrated and described by a number ofpatentable categories or conditions including any new and usefulcombinations of processes, machines, products or substances, or any newand useful improvements to them. Accordingly, various aspects of thepresent disclosure may be performed entirely by hardware, may beperformed entirely by software (including firmware, resident software,microcode, etc.), or may be performed by a combination of hardware andsoftware. The above hardware or software can be referred to as “datablock”, “module”, “engine”, “unit”, “component” or “system”. Inaddition, the aspects of the present disclosure may be embodied as acomputer product located on one or more computer readable media,including a computer readable program code.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including electro-magnetic, optical, or thelike, or any suitable combination thereof. A computer readable signalmedium may be any computer readable medium that is not a computerreadable storage medium and that may communicate, propagate, ortransport a program for use by or in connection with an instructionexecution system, apparatus, or device. Program code embodied on acomputer readable signal medium may be transmitted using any appropriatemedium, including wireless, wireline, optical fiber cable, RF, or thelike, or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of thepresent disclosure may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB. NET,Python or the like, conventional procedural programming languages, suchas the “C” programming language, Visual Basic, Fortran 1703, Perl, COBOL1702, PHP, ABAP, dynamic programming languages such as Python, Ruby andGroovy, or other programming languages. The program code may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider) or in a cloud computing environment or offered as aservice such as a software as a service (SaaS).

Furthermore, the recited order of processing elements or sequences, orthe use of numbers, letters, or other designations, therefore, is notintended to limit the claimed processes and methods to any order exceptas may be specified in the claims. Although the above disclosurediscusses through various examples what is currently considered to be avariety of useful embodiments of the disclosure, it is to be understoodthat such detail is solely for that purpose, and that the appendedclaims are not limited to the disclosed embodiments, but, on thecontrary, are intended to cover modifications and equivalentarrangements that are within the spirit and scope of the disclosedembodiments.

Similarly, it should be appreciated that in the foregoing description ofembodiments of the present disclosure, various features are sometimesgrouped together in a single embodiment, figure, or description thereoffor the purpose of streamlining the disclosure aiding in theunderstanding of one or more of the various embodiments. This method ofdisclosure, however, is not to be interpreted as reflecting an intentionthat the claimed subject matter requires more features than areexpressly recited in each claim. Rather, claimed subject matter may liein less than all features of a single foregoing disclosed embodiment.

In some embodiments, the numbers expressing quantities or propertiesused to describe and claim certain embodiments of the application are tobe understood as being modified in some instances by the term “about,”“approximate,” or “substantially.” For example, “about,” “approximate,”or “substantially” may indicate ±20% variation of the value itdescribes, unless otherwise stated. Accordingly, in some embodiments,the numerical parameters set forth in the written description andattached claims are approximations that may vary depending upon thedesired properties sought to be obtained by a particular embodiment. Insome embodiments, the numerical parameters should be construed in lightof the number of reported significant digits and by applying ordinaryrounding techniques. Notwithstanding that the numerical ranges andparameters setting forth the broad scope of some embodiments of theapplication are approximations, the numerical values set forth in thespecific examples are reported as precisely as practicable.

In closing, it is to be understood that the embodiments of theapplication disclosed herein are illustrative of the principles of theembodiments of the application. Other modifications that may be employedmay be within the scope of the application. Thus, by way of example, butnot of limitation, alternative configurations of the embodiments of theapplication may be utilized in accordance with the teachings herein.Accordingly, embodiments of the present application are not limited tothat precisely as shown and described.

We claim:
 1. A method, composing: obtaining image data; reconstructingan image based on the image data, wherein the image includes one or morefirst edges; obtaining a model, wherein the model includes one or moresecond edges corresponding to the one or more first edges; matching,based on the one or more first edges and the one or more second edges,the model and the image; and adjusting the one or more second edges ofthe model based on the one or more first edges.
 2. The method of claim1, wherein the image data includes a brain image, a skull image, a chestimage, a cardiac image, a breast image, an abdominal image, a kidneyimage, a liver image, a pelvic image, a perineal image, a limb image, aspine image, or a vertebra image.
 3. The method of claim 1, wherein theobtaining the model includes: obtaining a plurality of reference models;registering the plurality of reference models; determining a pluralityof control points on the plurality of reference models after theregistration; obtaining a control point of the model based on theplurality of control points on the plurality of reference models; andgenerating the model based on the control point of the model.
 4. Themethod of claim 3, further comprising: generating a correlation factorof the control point of the model based on a relationship between thecontrol point of the model and the one or more second edges of themodel.
 5. The method of claim 1, wherein the adjusting the one or moresecond edges of the model includes: determining a reference point on thesecond edge; determining an object point corresponding to the referencepoint; and adjusting the one or more second edges of the model based onthe object point.
 6. The method of claim 5, wherein the determining theobject point corresponding to the reference point includes: determininga normal of the reference point; obtaining a step size and a searchrange; determining one or more candidate points along the normal basedon the step size and the search range; obtaining a first classifier;determining a probability that the one or more candidate pointscorrespond to the first edge based on the first classifier; anddetermining the object point based on the probability that the one ormore candidate points correspond to the first edge.
 7. The method ofclaim 6, wherein the determining the normal of the reference pointincludes: determining one or more polygon grids adjacent to thereference point; determining one or more normals corresponding to theone or more polygon grids; and determining the normal of the referencepoint based on the one or more normals.
 8. The method of claim 4,wherein the adjusting the one or more second edges of the modelincludes: performing a similarity transformation on the one or moresecond edges; performing an affine transformation on the one or moresecond edges based on the correlation factor; or fine-tuning the one ormore second edges based on an energy function.
 9. The method of claim 1,wherein the matching the model and the image includes: obtaining asecond classifier; performing a weighted Generalized Hough Transformbased on the second classifier; and matching the model and the imagebased on a result of the weighted Generalized Hough Transform.
 10. Themethod of claim 6, wherein the obtaining the first classifier includes:obtaining a point classifier, wherein the point classifier classifies aplurality of points of the first edge based on image features related tosharpness and location; obtaining the plurality of classified points bythe point classifier, wherein at least a portion of the plurality ofclassified points are within a certain range of the first edge;determining a plurality of classified points within the certain range ofthe first edge as positive samples; determining a plurality ofclassified points outside the certain range of the first edge asnegative samples; classifying the positive samples and the negativesamples; and obtaining the first classifier based on the classifiedpositive samples and the classified negative samples.
 11. The method ofclaim 9, wherein the obtaining the second classifier includes: obtaininga plurality of points of the model, wherein at least a portion of theplurality of points are within a certain range of the second edge;determining a plurality of points within the certain range of the secondedge as positive samples; determining a plurality of points outside thecertain range of the second edge as negative samples; classifying thepositive sample and the negative sample based on the sharpness and thelocation; and obtaining the second classifier based on the classifiedpositive samples and the classified negative samples.
 12. A system,composing: a storage configured to store data and instructions; aprocessor in communication with the storage, wherein when executing theinstructions in the storage, the processor is configured to: obtainimage data; reconstruct an image based on the image data, wherein theimage includes one or more first edges; obtain a model, wherein themodel includes one or more second edges corresponding to the one or morefirst edges; match, based on the one or more first edges and the one ormore second edges, the model and the image; and adjust the one or moresecond edges of the model based on the one or more first edges.
 13. Thesystem of claim 12, wherein the processor is further configured to:obtain a plurality of reference models; register the plurality ofreference models; determine a plurality of control points on theplurality of reference models after the registration; obtain a controlpoint of the model based on the plurality of control points on theplurality of reference models; and generate the model based on thecontrol point of the model.
 14. The system of claim 13, wherein theprocessor is further configured to: generate a correlation factor of thecontrol point of the model based on a relationship between the controlpoint of the model and the one or more second edges of the model. 15.The system of claim 12, wherein the processor is further configured to:determine a reference point on the second edge; determine an objectpoint corresponding to the reference point; and adjust the one or moresecond edges of the model based on the object point.
 16. The system ofclaim 15, wherein the processor is further configured to: determine anormal of the reference point; obtain a step size and a search range;determine one or more candidate points along the normal based on thestep size and the search range; obtain a first classifier; determine aprobability that the one or more candidate points correspond to thefirst edge based on the first classifier; and determine the object pointbased on the probability that the one or more candidate pointscorrespond to the first edge.
 17. The system of claim 14, wherein theprocessor is further configured to: perform a similarity transformationon the one or more second edges; perform an affine transformation on theone or more second edges based on the correlation factor; or fine-tunethe one or more second edges based on an energy function.
 18. The systemof claim 12, wherein the processor is further configured to: obtain asecond classifier; perform a weighted Generalized Hough Transform basedon the second classifier; and match the model and the image based on aresult of the weighted Generalized Hough Transform.
 19. The system ofclaim 16, wherein the processor is further configured to: obtain a pointclassifier, wherein the point classifier classifies a plurality ofpoints of the first edge based on image features related to sharpnessand location; obtain the plurality of classified points by the pointclassifier, wherein at least a portion of the plurality of classifiedpoints are within a certain range of the first edge; determine aplurality of classified points within the certain range of the firstedge as positive samples; determine a plurality of classified pointsoutside the certain range of the first edge as negative samples;classify the positive samples and the negative samples; obtain the firstclassifier based on the classified positive samples and the classifiednegative samples.
 20. A non-transitory computer-readable medium havingcomputer programs, the computer programs comprising instructions,wherein the instructions are executed by at least one processor toperform a method, the method comprising: obtaining image data;reconstructing an image based on the image data, wherein the imageincludes one or more first edges; obtaining a model, wherein the modelincludes one or more second edges corresponding to the one or more firstedges; matching, based on the one or more first edges and the one ormore second edges, the model and the reconstructed image; and adjustingthe one or more second edges of the model based on the one or more firstedges.